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IMAGE DATA BASED RETROSPECTIVE TEMPORAL SELECTION OF MEDICAL 

IMAGES 



Field 

5 The present invention relates generally to medical imaging, and more particularly to 

systems and methods for performing temporal selection of medical images based on image 
data. 

Copyright Notice/Permission 

10 A portion of the disclosure of this patent document contains material that is subject to 

copyright protection. The copyright owner has no objection to the facsimile reproduction by 
anyone of the patent document or the patent disclosure as it appears in the Patent and 
Trademark Office patent file or records, but otherwise reserves all copyright rights 
whatsoever. The following notice applies to the software and data as described below and in 

15 the drawings hereto: Copyright © 2000, Vital Images, Inc. All Rights Reserved. 

Background 

Heart disease is a significant public health problem. Therefore, in medicine, it is of 
considerable value to obtain cross-sectional and volumetric images of the human heart. 
20 Applications include angiography of coronary vessels for detecting stenosis, examination of 
the thoracic aorta for dissection of the vessel wall, and quantization of calcium deposits in the 
coronary arteries as a marker for atheroscerotic plaque. 
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Recently, this last application has gathered significant interest due to the finding that 
the exam can be performed with a conventional high-speed helical CT (Computer 
Tomography) scanner, as opposed to the more expensive and less prevalent electron beam CT 
scanner. However, helical CT scanners are still too slow to "freeze" the motion of the heart, 
5 resulting in some cross-sectional images being blurred and out of register with other images 
captured at a different phase of the heart's cycle of contraction (systole) and relaxation 
(diastole). 

As a consequence, methods have been introduced to determine the heart cycle phase of 
axial CT images by simultaneously recording the electrocardiogram (EKG), and 

10 synchronizing this signal with the sequence of images. Since the high amplitude "r-wave" of 
the EKG is a reliable and very brief indicator of the onset of ventricular contraction, it is 
straightforward to use its peak as an indicator of when heart motion will be near its most 
violent. It is then possible to either shut down image acquisition during these times, or 
retrospectively eliminate images that have been acquired at these times. This method is called 

15 EKG gating. 

The disadvantage of EKG gating is that it significantly lengthens the exam time due to 
the need to wire the patient with electrodes, etc. It also introduces additional complications in 
the design of the CT scanner and of the software used to post-process the acquired images. It 
may additionally require manual verification of the selection of images, and introduces the 
20 possibility of mismatching EKG traces and images from different patients. 
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As a result, there is a need in the art for systems and methods that can filter images 
acquired during an image scan of a patient without the need for external monitoring devices 
such as EKG monitors. 



5 Summary 

The above-mentioned shortcomings, disadvantages and problems are addressed by the 
present invention, which will be understood by reading and studying the following 
specification. 

In one embodiment of the invention, a method for selecting images of a portion of a 
10 cardiovascular system includes receiving a plurality of images from a scanner that have been 
recorded over a period of time. The images represent one or more locations along the extent 
of the cardiovascular system. The images are then selected based on common criteria 
determined from the plurality of images and without reference to an external signal. 

In some embodiments, the common criteria comprises changes in the size of a cross 
15 section of the aorta. In alternative embodiments, the common criteria comprises changes in 
the volume of the heart. In still further embodiments, the common criteria comprises changes 
in the area of a cross section of the heart. In yet other embodiments, the criteria includes the 
mean pixel difference between adjacent images. 

The present invention describes systems, clients, servers, methods, and computer- 
20 readable media of varying scope. In addition to the aspects and advantages of the present 
invention described in this summary, further aspects and advantages of the invention will 
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become apparent by reference to the drawings and by reading the detailed description that 
follows. 

Brief Description Of The Drawings 

5 FIG. 1 is a block diagram of the hardware and operating environment in which different 
embodiments of the invention can be practiced; 
FIG. 2 is a diagram illustrating a system level overview of an exemplary embodiment of the 
invention; 

FIG. 3 is a flowchart illustrating a method for performing retrospective gating of medical 
10 image data according to an exemplary embodiment of the invention; 

FIG. 4 is a flowchart providing further details on determining a signal used to filter images; 
and 

FIG. 5 provides a graph comparing the image based retrospective gating to prior art EKG 
gating. 

15 

Detailed Description 

In the following detailed description of exemplary embodiments of the invention, 
reference is made to the accompanying drawings which form a part hereof, and in which is 
20 shown by way of illustration specific exemplary embodiments in which the invention may be 
practiced. These embodiments are described in sufficient detail to enable those skilled in the 
art to practice the invention, and it is to be understood that other embodiments may be utilized 
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and that logical, mechanical, electrical and other changes may be made without departing 
from the scope of the present invention. The following detailed description is, therefore, not 
to be taken in a limiting sense. 

In the Figures, the same reference number is used throughout to refer to an identical 
5 component which appears in multiple Figures. Signals and connections may be referred to by 
the same reference number or label, and the actual meaning will be clear from its use in the 
context of the description. 

The detailed description is divided into multiple sections. In the first section the 
hardware and operating environment of different embodiments of the invention is described. 
10 In the second section, the software environment of varying embodiments of the invention is 
described. In the third section, methods of various embodiments of the invention are 
described. In the final section, a conclusion is provided. 



Hardware and Operating Environment 
15 FIG. 1 is a diagram of the hardware and operating environment in conjunction with 

which embodiments of the invention may be practiced. The description of FIG. 1 is intended 
to provide a brief, general description of suitable computer hardware and a suitable computing 
environment in conjunction with which the invention may be implemented. Although not 
required, the invention is described in the general context of computer-executable instructions, 
20 such as program modules, being executed by a computer, such as a personal computer, 

workstation, or a server computer. Generally, program modules include routines, programs, 
objects, components, data structures, etc., that perform particular tasks or implement 
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particular abstract data types. 

Moreover, those skilled in the art will appreciate that the invention may be practiced 
with other computer system configurations, including hand-held devices, multiprocessor 
systems, microprocessor-based or programmable consumer electronics, network PCs, 

5 minicomputers, mainframe computers, and the like. The invention may also be practiced in 
distributed computing environments where tasks are performed by remote processing devices 
that are linked through a communications network. In a distributed computing environment, 
program modules may be located in both local and remote memory storage devices. 

As shown in FIG. 1, the computing system 100 includes a processor. The invention 

10 can be implemented on computers based upon microprocessors such as the PENTIUM 
family of microprocessors manufactured by the Intel Corporation, the MIPS® family of 
microprocessors from the Silicon Graphics Corporation, the POWERPC® family of 
microprocessors from both the Motorola Corporation and the IBM Corporation, the 
PRECISION ARCHITECTURE® family of microprocessors from the Hewlett-Packard 

15 Company, the SPARC® family of microprocessors from the Sun Microsystems Corporation, 
or the ALPHA® family of microprocessors from the Compaq Computer Corporation. 
Computing system 200 represents any personal computer, laptop, server, or even a battery- 
powered, pocket-sized, mobile computer known as a hand-held PC. 

The computing system 100 includes system memory 113 (including read-only memory 

20 (ROM) 1 14 and random access memory (RAM) 1 15), which is connected to the processor 112 
by a system data/address bus 1 16. ROM 1 14 represents any device that is primarily read-only 
including electrically erasable programmable read-only memory (EEPROM), flash memory, 
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etc. RAM 1 15 represents any random access memory such as Synchronous Dynamic Random 
Access Memory. 

Within the computing system 100, input/output bus 1 18 is connected to the 
data/address bus 1 16 via bus controller 1 19. In one embodiment, input/output bus 1 18 is 

5 implemented as a standard Peripheral Component Interconnect (PCI) bus. The bus controller 
1 19 examines all signals from the processor 1 12 to route the signals to the appropriate bus. 
Signals between the processor 1 12 and the system memory 1 13 are merely passed through the 
bus controller 1 19. However, signals from the processor 1 12 intended for devices other than 
system memory 1 13 are routed onto the input/output bus 118. 

1 0 Various devices are connected to the input/output bus 1 1 8 including hard disk drive 

120, floppy drive 121 that is used to read floppy disk 151, and optical drive 122, such as a 
CD-ROM drive that is used to read an optical disk 152. The video display 124 or other kind 
of display device is connected to the input/output bus 1 18 via a video adapter 125. 

A user enters commands and information into the computing system 100 by using a 

15 keyboard 40 and/or pointing device, such as a mouse 42, which are connected to bus 1 1 8 via 
input/output ports 128. Other types of pointing devices (not shown in FIG. 1) include track 
pads, track balls, joy sticks, data gloves, head trackers, and other devices suitable for 
positioning a cursor on the video display 124. 

As shown in FIG. 1, the computing system 100 also includes a modem 129. Although 

20 illustrated in FIG. 1 as external to the computing system 100, those of ordinary skill in the art 
will quickly recognize that the modem 129 may also be internal to the computing system 100. 
The modem 129 is typically used to communicate over wide area networks (not shown), such 
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as the global Internet. The computing system may also contain a network interface card 53, as 
is known in the art, for communication over a network. 

Software applications 136 and data are typically stored via one of the memory storage 
devices, which may include the hard disk 120, floppy disk 151, CD-ROM 152 and are copied 
5 to RAM 115 for execution. In one embodiment, however, software applications 136 are 
stored in ROM 114 and are copied to RAM 115 for execution or are executed directly from 
ROM 114. 

In general, the operating system 135 executes software applications 136 and carries out 
instructions issued by the user. For example, when the user wants to load a software 

10 application 136, the operating system 135 interprets the instruction and causes the processor 
1 12 to load software application 136 into RAM 1 15 from either the hard disk 120 or the 
optical disk 152. Once software application 136 is loaded into the RAM 1 15, it can be used by 
the processor 1 12. In case of large software applications 136, processor 1 12 loads various 
portions of program modules into RAM 1 15 as needed. 

15 The Basic Input/Output System (BIOS) 1 17 for the computing system 100 is stored in 

ROM 1 14 and is loaded into RAM 1 15 upon booting. Those skilled in the art will recognize 
that the BIOS 1 17 is a set of basic executable routines that have conventionally helped to 
transfer information between the computing resources within the computing system 100. 
These low-level service routines are used by operating system 135 or other software 

20 applications 136. 

In one embodiment computing system 100 includes a registry (not shown) which is a 
system database that holds configuration information for computing system 100. For 



example, Windows® 95 , Windows 98®, Windows® NT, and Windows 2000 by Microsoft 
maintain the registry in two hidden files, called USER.DAT and SYSTEM.DAT, located on a 
permanent storage device such as an internal disk. 

5 Software Environment 

The embodiments of the invention describe a software environment of systems and 
methods that provide for the retrospective gating of medical images. FIG. 2 is a block 
diagram describing the major components of such a system. As shown, the system includes 
an image scanner 202 and an image processing system 204. 

L0 Image scanner 202 in one embodiment of the invention is a CT scanner. The scanner 

can be a high-speed helical CT scanner, or it can be an electron beam CT scanner. However, 
the invention is not limited to CT scanners, an any scanner that can provide a sequence of 
images recorded over a period of time are within the scope of the invention. For example, 
scanner 202 could be a Magnetic Resonance Imaging (MRI) or ultrasound scanner. 

1 5 Scanner 202 produces image data 204 that comprises a sequence of two-dimensional 

images of the human body. This image data is then sent to image processing system 206 for 
processing. Li one embodiment of the invention, image processing system 206 is the 
ImageGate system from Vital Images, Inc. The image data can be transferred from scanner 
202 to image processing system 204 using any data transmission means, including tape media, 

20 CD-ROM, floppy-disk, removable hard drive, and network means, including the Internet. 

Image processing system 206 is a suitably configured computer, such as the computer 
illustrated in FIG. 1, and employs the methods detailed below to perform retrospective gating 
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of the image data. The output of system 206 comprises cardiac cycle signal 208 and filtered 
image data 210. Filtered image data 210 comprises the image data that corresponds to images 
acquired at desired points in the cardiac cycle signal 208. 

This section has described the various system components in a system that performs 
5 image based retrospective gating of cardiac images. As those of skill in the art will 

appreciate, the software can be written in any of a number of programming languages known 
in the art, including but not limited to C/C++, Visual Basic, Smalltalk, Pascal, Ada and 
similar programming languages. The invention is not limited to any particular programming 
language for implementation. 

10 

Methods of an Exemplary Embodiment of the Invention 
In the previous section, a system level overview of the operation of an exemplary 

embodiment of the invention was described. In this section, the particular methods of various 

embodiments of the invention performed by an operating environment executing an 

15 exemplary embodiment are described by reference to a flowchart shown in FIGs. 3 and 4. The 
methods to be performed by the operating environment constitute computer programs made 
up of computer-executable instructions. Describing the methods by reference to a flowchart 
enables one skilled in the art to develop such programs including such instructions to carry out 
the methods on suitable computers (the processor of the computer executing the instructions 

20 from computer-readable media). The methods illustrated in FIGs. 3 and 4 are inclusive of the 
acts required to be taken by an operating environment executing an exemplary embodiment of 
the invention. 
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The methods of the various embodiments of the invention illustrated below operate to 
perform time categorization and selection of a subset of images from a plurality of images 
based on the image data alone, that is, without reference to any external signals such as an 
EKG. In some embodiments of the invention, the plurality of images comprise a series of 
images that are received from a CT scanning system in which multiple images are taken at 
different points in time of a single space of a body. In other embodiments, the plurality of 
images can comprise a sequence of images in which one image of a space of a body at a single 
point in time is taken. 

A method for categorizing and selecting images according to an embodiment of the 
invention for performing image-based retrospective gating of scanned medical image data is 
illustrated in FIG. 3. A system executing the method begins by receiving scanned image data 
(block 302). In one embodiment of the invention, the scanned image data is received from a 
CT scanner such as scanner 202 (FIG. 1). However, the invention is not limited to any 
particular method of obtaining the scanned image data. As will be appreciated by those of 
skill in the art, any system capable of producing a volumetric sequence of images is within the 
scope of the invention. It is desirable that the images are heavily overlapped axial images of 
the chest. The axial overlap provides the ability for the system to select a subset of images 
acquired when the heart was most at rest and still adequately sample the heart to its full 
anatomical extent. 

In one embodiment of the invention, the categorization and selection of images 
operates to select those images that are the least blurred. In the case of cardiac image data, the 
least blurred images represent those points in time in the cardiac cycle when the heart is at rest 

11 



in between beats. In some embodiments of the invention, a Fourier transform is used to 

determine "blurriness" of an image. 

In alternative embodiments, the Mean Pixel Difference (MPD) between one image and 

a subsequent adjacent image is determined. The images with the lowest MPD are selected as 
5 being the least blurry images. In MPD, each pixel on a first axial image is subtracted from the 

corresponding pixel on the subsequent adjacent axial image. The absolute value of the pixel 

differences for the image are summed and divided by the number of pixels in the image. This 

single number for each image is the MPD. An image with a lower MPD than another image 

can be said to be less blurry. 
10 Next, those images that are the least blurry, or whose blurriness are within a particular 

tolerance, are then selected for inclusion in the set of images for which further analysis, such 

as volumetric analysis, will be performed. 

The image processing system derives a cardiac cycle signal from the image data (block 

304). The various phases of the cardiac cycle can be derived from the image data by detecting 
1 5 in the image data the corresponding motion or changes of different parts of the heart and/or 

other parts of the body affected by the motion of the heart, such as blood vessels. By 

measuring this motion as a function of a time, a periodic signal can be derived that can be 

used to determine heart rate and cardiac cycle phase for any time during the image data 

collection. 

20 Finally, the image data is filtered according to the cardiac cycle signal determined at 

block 304 to select those images that where acquired when the heart was most at rest (block 
306). The filtered images can then be used to render accurate volumetric images of the heart. 
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Various methods can be used to determine the cardiac cycle in block 304 above. The 
methods generally determine the cycle based on criteria common to the images. One such 
method according to an embodiment of the invention detects changes in the cross-section of 
the aorta, and is illustrated in FIG. 4. The method begins by estimating the position of the 
5 aorta (block 402). Various means can be used to detect the position of the aorta. For 
example, in some embodiments of the invention, a matching filter as is known in the art is 
used to detect the aorta. In alternative embodiments, an Active Appearance Model is used to 
detect the position. In further alternative embodiments, an Active Shape Model is used. 

In still further embodiments of the invention, an algorithm called Hough Transform is 

10 used to find the approximate aorta position. As is known in the art, the Hough Transform can 
be regarded as a generalized template matching method, and is typically used to extract out 
edges or curves from an image. The Hough Transform can be used to extract circles and even 
generalized (perhaps non-symmetrical) edges. Use of the Hough Transform is desirable, 
because it is invariant to rotation and translation. Further details on the Hough Transform can 

15 be found in U.S. Patent 3,069,654, which is hereby incorporated by reference herein. For the 
purpose of Hough Transform, aorta can be described as a circle of radius in a range 8 - 25mm. 

In one embodiment of the invention, the position of the aorta is determined only in the 
first image. For the following images, the results of segmentation in the previous image are 
used. This is desirable because it reduces the time required to execute the method. The 

20 approximate aorta border in the (z + 1) image is defined as a circle of radius Rres, where Rres 
is the radius of the circle found by Hough transform in image z. The circle is centered at 
position (Xc(z), Yc(z)), which is the center of aorta found in the previous, z image. Because 
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processing each image is a sequential process, it is vulnerable to error propagation. In one 
embodiment of the invention, the likelihood of possible error propagation is reduced by using 
the same circle radius to initialize the aorta segmentation and only updating its position in 
subsequent images. 

5 In another embodiment of the invention, a voting mechanism for detecting the aorta is 

used. The Hough Transform is applied to a sequence of five consecutive images. A result is 

accepted, if the aorta center is found in a similar location in at least four of these images. If 

such a match is not found, another set of five images is investigated. 

In the Hough transform, a fit value is determined for every pixel in an image. The fit 

10 value represents the probability that the point belongs to the aorta border. The computation of 

fit values is based on a priori knowledge of the aorta properties in CT images. 

For every pixel (x, y) in an image, the fit value is computed as 

_ JO if /(*, y) < -10OHU or J(x, y) > 200HU 
//f(X ^ ) = |/b(x,y)H expected ] otherwise 



15 where: 

I(x, y) is the image intensity in Hounsfield Units [HU]. 
g is the image gradient 

n ted is the expected direction of the gradient. 
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In one embodiment of the invention, a directional gradient detector based on 5 x 5 
Gaussian mask is used. In the case of the Hough Transform, the expected direction is not 
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known. Therefore, one embodiment of the invention uses the following formula to set 

fi 

expected • 

g 

n expected = TZ\ (2) 

Function f(.) in formula (1) is a transformation function that limits the maximum value 
of the considered gradient. The transformation function f(.) in one embodiment of the 
invention is: 

{a:a< a MAY 
a MAX ' a — a MAX 

where a MAX is the gradient response to a step edge of volume 100HU. 

After estimating the position of the aorta, the method then segments (i.e. detects) the 
aorta in each of the 2D images in the sequence (block 404). In some embodiments of the 
invention, edge-based dynamic programming as is known in the art is applied to the images in 
order to segment the aorta. In alternative embodiments, Active Contours, also referred to as 
"Snakes" are used to detect the aorta. In further embodiments of the invention, region 
growing algorithms are used to detect the aorta. Such algorithms are known in the art. 

In one particular embodiment of the invention, the aorta is segmented using a dynamic 
programming based boundary detection algorithm. Details of such a boundary detection 
algorithm are presented in M. Sonka, V. Hlavac, R. Boyle, Image Processing, Analysis, and 
Machine Vision; PWS Publishing, 2 nd edition, 1998, pp. 158-163, which is hereby 
incorporated by reference herein. The approximate aorta location determined at block 402 is 
used to create a Region of Interest (ROI) within which the aorta boundary is sought. For the 
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purpose of searching the image for the aorta, the ROI is mapped into a rectangular graph, in 
which every node (xg, Yg) corresponds to one image pixel. This is expressed in the following 
formula: 

(*g > >'g ) = centerlinePoint(x G ) + n c (x G )^ (y G - Wl ^ th ) (4) 

where: 

n c is the unit normal vector to the circular centerline at xoth point along the 

centerline; and 

width is the width of the ROL 

Every node in the rectangular graph has a cost associated with it. 
the formula: 

cost(x, y) = ma,x(fit(i, j) - fit(x, y)) 

where: 

fit is defined by Equation 1 with n exp ected = n c . 

The method finds a path of minimal cost between any node in the first column and any 
node in the last column of the rectangular graph. To ensure that the found boundary is a closed 
contour, the search is restricted by forcing the first and last points of the minimal cost path 
have to have the same Yg coordinate in the rectangular graph. 

In addition to assigning a cost to every graph node, a cost is assigned to every link 
between graph nodes. Only nodes in 2 consecutive columns are connected by links. The costs 
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The cost is given by 



(5) 



are assigned according to the following formula: 



cost[(x,3v),(x + l,y*)] : 



0 ifk = j 

0.3 • cost(x;, yj) if k- j±l 
oo otherwise 



(6) 



5 Such a cost assignment effectively connects only nodes whose y coordinate differs by 

no more than one. It ensures connectivity of the resulting boundary. The middle rule in 
Equation 6 for assigning cost reflects the knowledge that the sought border should be circular. 
Thus, a border that is parallel to the ROI centerline (a circle) is considered to be a preferred 
direction over a direction that diverges from the ROI centerline direction. The minimum cost 

10 path is then mapped into the original image and represents the aorta border. 

After determining the position of the aorta, the method then computes the area of the 
aorta cross-section in the image (block 406). The cross-sectional area in the image is then 
used to compute an approximation of the volume of aorta at the time of an image acquisition. 
The area is computed as an area enclosed by the aorta border found by the aorta segmentation 

15 in block 404. In one embodiment of the invention, for every image, the found border is 
projected onto a plane that is perpendicular to the aorta direction. This provides better 
approximation of the true cross-sectional area of the aorta in cases when the image acquisition 
plane is not perpendicular to the aorta direction. 

The aorta direction for a given image can be approximated by a tangent to a line 

20 connecting centers of gravity of outlined aorta boundaries. In some embodiments of the 
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invention, the line is smoothed using algorithms known in the art. 

Every border point (x, y, z) is projected into a point (x P , y P , z P ) as follows: 
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where: 



t is the unit vector representing aorta direction; and 

c is a vector connecting the boundary point (x, y, z) with the aorta 

center in the given image. 



The aorta center can be computed from its boundary points, yielding the following 
10 computation of the vector c : 
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where: 

(xi ,yi ,zd are the aorta boundary points 
15 N B is the total number of boundary points. 



In some embodiments of the invention, the area computation is simplified by 
expressing the projected border in a new coordinate system (w x , w y , w z ) in which the w z 
coordinate of all the boundary points is zero. The new coordinate system is determined as 
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follows: 



w. 
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e r -w z » (w z • e x ) - w y • (w y • e x ) 



(9) 



Thus, the coordinate transformation can be computed as: 



where: 
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Because z r = 0 for all points of the transformed border, in some embodiments the area 
10 of the aorta cross-section is computed using a discrete implementation of Simpson's Rule. 
Such algorithms are known in the art. 

The sequence of values representing the aorta cross-sectional areas represent an in- 
time regularly sampled signal that can be used to compute the heart rate and the phase of the 
cardiac sample for every image in the sequence. 
15 In some embodiments of the invention, the signal determined is filtered to remove 
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noise and to simplify cardiac phase determination (block 408). In some embodiments of the 
invention, filtering is performed by a Butterworth filter. It is desirable that the filter is 
designed so that there is no more than 3dB attenuation in the pass band and at least 40dB 
attenuation in the stop band. In order to determine the band pass of the filter, the aorta area 
5 signal is first High Pass Filtered (HPF) in order to remove DC component and low frequencies 
that may correspond to the change of aorta size in space rather than in time. In some 
embodiments, the HPF signal (S H pf) is computed using a digital high-pass filter based on the 
Butterworth filter. 

Next, the heart rate is determined. In some embodiments, it is estimated by 
10 determining the main frequency component in the area signal. It is desirable to avoid 

rasterization and to ensure stability, thus some embodiments determine the main frequency 
not from the original signal, but from the signal's autocorrelation: 

/max = ar § max\\Fourier{sHPF * sHPF}\\ (12) 
where * denotes the correlation operation. 
15 The raw aorta area signal is then filtered with a Butterworth filter with a passing band 

of < 0.7/, 13 f > , which represents an estimated variation of the heart rate for a single patient 
during the scanning procedure. In order to improve the filtering results, some embodiments 
use the signal obtained using the above-described filtering to compute new values of f L and f H . 
The beginning of each period is determined by positively sloped zero crossings of the signal 
20 The newly obtained limits are used to finally filter the signal according to the following 
formula: 
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S BPF (jc) = Butterworth{s(x\ f L ,f H }, 
where: 

s denotes the raw aorta area signal; and 

< Sl > Sh > is the frequency range of the pass band. (13) 
In alternative embodiments, signal filtering is performed by applying an alternative 
HPF to the aorta signal, again, in order to remove DC component and low frequencies that 
may correspond to change of aorta size in space rather than in time. In these embodiments, 
the HPF can be approximated by subtracting a moving average from the original signal 
according to the following: 

S HPF (x) = s(x)-^s(x + i) (14) 

i=-7 

where s denotes the raw aorta area signal. 

Next, in order to remove noise from the signal and thus simplify the determination of 
points of maximum positive gradient that corresponds to the opening of aortic valve, the 
signal is smoothed. 

In alternative embodiments of the invention, smoothing is performed using Band Pass 
Filtering (BPF) in a frequency domain. A moving window Fourier Transform can be used to 
smooth the signal separately in short segments. The window width is chosen so that it 
corresponds to approximately three cardiac cycles and is a power of 2 (to simplify the 
employment of Fast Fourier Transform algorithm). The width of the BPF is chosen to be one, 
thus considering only the basic frequency that occurs in the given segment of the signal. The 
BPF frequency is selected as the highest frequency in the spectrum (not considering zero 
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frequency - the signal offset). The window for the Fourier Transform is always moved by just 
one sample and the smoothed signals are averaged in the time domain. The filtering is 
illustrated in the following formula: 

5 ^(*<,v +w >) = Fourier" 1 {Fourier{ S<i . +w> m Filter, .}} (15) 
where: 

Filter is a BPF with a passing frequency: 

fi = arg ^max 2> Fourier{s <i4+w> } (16) 
and the window width w is a power of 2. 

The final filter signal is thus: 



X BPF(S <jJ+w> )(i-j)- 1 



j=max(i-w,o) mm(z, N - w) - max(z - w,0) 



(17) 



where N is the number of samples in the aorta area signal. 



After the signal has been filtered, the method determines the cardiac phase (block 
410). Depending on the heart rate of the person being scanned and on the image acquisition 
speed, 6-12 images can typically be obtained per cardiac cycle. It is generally the case that the 
opening of the aortic valve is followed by the maximum rate of change (max. gradient) of 
aortic pressure that is reflected by the maximum gradient of increase of aorta cross-sectional 
area. This happens approximately 0. 1 seconds after the beginning of ventricular polarization. 
To match the aorta area signal to the heart cycle the method searches for the points of 
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maximum positive gradient, which generally correspond to the moments of aortic valve 
opening. 

The filtered signal approximates the time change of aorta cross-sectional area 
(volume) during a cardiac cycle. Thus, one period of the signal corresponds to one cardiac 
5 cycle. Since signal can be approximated by a sine wave, the maximum positive derivative of 
the signal can be determined and used as the point of a positively sloped zero crossing. Using 
linear interpolation, the method can determine that the zero crossing point x z between signal 
points (x, y\) and (x + 1, y 2 ), such that is: 

10 + * (18) 

yx-yi 

Thus the set of times (x z ) represents the instants corresponding to aortic valve 
openings. From the aortic valve openings, the point in the cardiac cycle signal can be 
determined where the heart is most at rest. 

1 5 In order to remove possible errors in the signal filtering, some embodiments of the 

invention perform post-processing on the signal. Based on the derived heart rate statistics 
(mean and standard deviation), outliers representing too short heart rate can be removed 
and/or a heart cycle can be added if the measured one is too long. 

The above-described signal filtering techniques can be used for both helical scanning 

20 systems and for multi-slice scanning systems. In the case of multi-slice scanning systems, the 
scanning protocol is generally a vv step and shoot" procedure, which results in a signal 
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consisting of short measured pieces (1-2 heart cycles) interlaced with segments of 1-2 heart 
cycles without any data. In such systems there are typically more than one measurement of 
every segment. Usually there are four segments, and the segments are synchronized. In order 
to use the same filtering scheme as above, the existing signals are averaged to obtain one 
signal, resulting in a higher S/N ratio. Then, the missing segments of the signal are 
reconstructed. A method for restoration of lost samples in digital signals is described in detail 
in Raymond Veldhuis: Restoration of Lost Samples in Digital Signals. Prentice Hall 
International Series in Acoustics, Speech and Signal Processing, 1990 Prentice Hall 
International (UK) Ltd., ISBN - 0-13-775198-2, Chapter 3 Autoregressive processes, pp. 28- 
56. In some embodiments, this algorithm is used to fill the missing data. However, the 
invention is not limited to the algorithm described in Veldhuis, and any algorithm capable of 
restoring lost samples can be used. When the signal is reconstructed to its full length, the 
same filtering scheme as described above can be employed 

The method of determining the cardiac cycle illustrated in FIG. 4 is desirable, for 
several reasons. First, it is easy to measure accurately because a reliable segmentation 
algorithm exists. Second, measuring the cross section of the aorta is insensitive to the shifting 
and twisting that occurs as the heart beats. Finally, measuring the cross section of the aorta is 
independent of heart geometry changes in the space domain (changes from one acquired 
image to another). However, the invention is not limited to deriving a cardiac cycle signal 
based on the changes in the area of the cross section of the aorta. For example, changes in the 
area of cross section of other blood vessels besides the aorta can be detected and used to 
derive a signal. In addition, the invention is not limited to detecting changes in the area of a 

24 



blood vessel such as the aorta. In alternative embodiments of the invention, the motion of a 
blood vessel wall is determined and used to derive a signal. 

FIG. 5 illustrates a sample of the signals obtained using the methods described above. 
Graph 502 illustrates the difference between an EKG-derived signal an image-derived signal 
5 for ventricular systole. Graph 504 illustrates the difference between an EKG-derived heart 
rate and an image-derived heart rate. Graph 506 presents two signals, Raw signal 508 which 
represents the signal before filtering, and filtered signal 510 which represents the signal after 
the filtering described above has been applied. 

In an alternative embodiment of the invention, the cardiac signal can be derived using 
10 a "Mean Pixel Difference" (MPD) between the acquired images. As noted above, in MPD, 
each pixel on a first axial image is subtracted from the corresponding pixel on the subsequent 
adjacent axial image. The absolute value of the pixel differences for the image are summed 
and divided by the number of pixels in the image. This single number for each image is the 
MPD. 

15 The MPD thus represents a direct measure of changes in the data between two image 

slices. The sequence of MPD values can thus be used to derive a cardiac cycle signal. Using 
the MPD is not as desirable as using the changes in the area of cross-sections of the aorta, 
because other factors besides heart motion can affect the calculation of the MPD. For 
example, the difference is often caused not only by the heart motion, but also by the change of 

20 the heart geometry as the scan progresses both in the time and spatial domains. 

In a further alternative embodiment of the invention, the cardiac signal can be derived 
using the fact that the heart volume periodically decreases and increases during systole and 
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diastole. By measuring these changes in heart volume (or area of heart cross-section) across a 
sequence of images, a cardiac cycle can be computed. The reliable measurement of the heart 
cross-sectional area requires accurate segmentation of the heart in each of the images. The 
cross-sectional area can then be used to determine a cardiac cycle signal in a manner similar to 
that described above in reference to FIG. 4. 

In a still further embodiment of the invention, heart border motion can be used to 
derive a cardiac cycle signal. Here, heart border motion is measured by detecting the motion 
in the walls of the heart. For example, the atrium and ventricle borders. This can provide the 
same information as measurement of heart cross-sectional area. In this embodiment, easy-to- 
segment parts of the heart are measured and the changes in the measurements used to derive a 
cardiac cycle signal. 

Conclusion 

Systems and methods for using image data to derive signals, such as cardiac cycle 
signals have been disclosed. The embodiments of the invention provide advantages over 
previous systems. For example, there is no need to attach wires to the patient for EKG, which 
can reduce patient apprehension and nervousness about the imaging procedure. Furthermore, 
deriving the cardiac cycle from the image data eliminates the possibility that the EKG cycle 
data does not match the image data due to mishandling of the EKG data, since no EKG data is 
required. Moreover, EKG data can provide a less than accurate cardiac cycle due to variations 
in the measurement of electrical signals generated by the heart. The image data captures the 
heart motion directly, unlike the EKG. 
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Although specific embodiments have been illustrated and described herein, it will be 
appreciated by those of ordinary skill in the art that any arrangement which is calculated to 
achieve the same purpose may be substituted for the specific embodiments shown. This 
application is intended to cover any adaptations or variations of the present invention. 

The terminology used in this application is meant to include all of these environments. 
It is to be understood that the above description is intended to be illustrative, and not 
restrictive. Many other embodiments will be apparent to those of skill in the art upon 
reviewing the above description. Therefore, it is manifestly intended that this invention be 
limited only by the following claims and equivalents thereof. 
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We claim: 

1 . A method for selecting images of a portion of a cardiovascular system comprising: 
receiving from an image scanner a plurality of images recorded over a period of time, 

the images representing one or more locations along the extent of the cardiovascular system; 
5 and 

selecting at least a subset of the images based on common criteria determined from the 
plurality of images and without reference to an external signal. 

2. The method of claim 1, wherein the portion of the cardiovascular system is the heart. 

10 

3. The method of claim 1, wherein the image scanner is a CT (Computer Tomography) 
scanner. 

4. The method of claim 1, wherein the image scanner is a MRI (Magnetic Resonance 
15 Image) scanner. 

5. The method of claim 1, wherein the image scanner is an Ultrasound scanner. 

6. The method of claim 1, wherein selecting a subset of the images results from a 
20 determination of the blurriness of each image. 
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7. The method of claim 6, wherein the blurriness of the image is determined by a Fourier 
transform applied to the image. 

8. The method of claim 6, wherein the blurriness of the image is determined by the mean 
pixel difference between the image and an adjacent image. 

9. The method of claim 1 , wherein selecting a subset of the images results from a 
determination of a change of a relative position of at least one vessel edge in each image, 

10. A method for ordering a plurality of images of a portion of a cardiovascular system 
comprising: 

receiving from an image scanner a plurality of images recorded over a period of time, 
the images representing one or more locations along the extent of the cardiovascular system; 
deriving a cardiac cycle signal from the plurality of scanned images; and 
assigning a phase in the cardiac cycle to each scanned image. 

1 1 . The method of claim 10, wherein the portion of the cardiovascular system is the heart. 

12. The method of claim 10, wherein the image scanner is a CT (Computer Tomography) 
scanner. 
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13. The method of claim 10, wherein the image scanner is a MRI (Magnetic Resonance 
Image) scanner. 

14. The method of claim 10, wherein the image scanner is an Ultrasound scanner. 

5 

15. The method of claim 10, wherein deriving the cardiac signal comprises: 
segmenting a set of data representing a blood vessel in each image; 
computing a change value for the blood vessel; and 

determining the cardiac cycle signal based on a sequence of the change values for each 
10 image. 

16. The method of claim 15, wherein the change value is a change in the area of a cross 
section of the blood vessel 

15 17. The method of claim 15, wherein the change value is a change in the position of a wall 
of the blood vessel. 

18. The method of claim 10, wherein deriving the cardiac cycle signal comprises: 
segmenting a set of data representing a cross-section of the aorta in each image; 

20 computing an area value representing an area of the cross-section; and 

determining the cardiac cycle signal based on a sequence of the area values for each 
image. 

19. The method of claim 18, further comprising estimating the position of the aorta within 
25 the image data prior to segmenting the set of data. 

20. The method of claim 19, wherein estimating the position of the aorta utilizes a Hough 
transform. 
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21 . The method of claim 18, wherein segmenting of the aorta cross-section utilizes 
Dynamic Programming. 

22. The method of claim 18, further comprising filtering the cardiac cycle signal to 
produce a smoothed cardiac cycle signal. 



23. The method of claim 10, wherein deriving the cardiac cycle signal comprises: 
10 for each image in the plurality of images performing the tasks of: 

selecting an adjacent subsequent image; 

calculating a mean pixel difference between the image and the subsequent 
image; and 

determining the cardiac cycle signal based on the mean pixel differences of the 

15 images. 

24. The method of claim 10, wherein deriving the cardiac cycle signal comprises: 
segmenting a set of data representing a cross-section of a heart in each image; 
computing an area value representing an area of the cross-section; and 

20 determining the cardiac cycle signal based on a sequence of the area values for each 

image. 

25. The method of claim 10, wherein deriving the cardiac cycle signal comprises: 
for each image in the plurality of images performing the tasks of: 

25 determining a first border of a heart in the image; 

determining a second border of the heart in a subsequent adjacent image; 
determining the difference between the first border and the second border; and 
determining the cardiac cycle based on a sequence of the differences. 
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26. The method of claim 10, wherein the ordered set of images is further 

filtered to produce a subset of images, said subset of images comprising images acquired at a 

desired point in the cardiac cycle signal. 

27. The method of claim 1 0, wherein the derived cardiac cycle signal is used to interpolate 
or reconstruct new images at specific phases in the cardiac cycle from the original scanned 
images or other related data. 

28. A computer-readable medium having computer executable instructions for performing 
a method for selecting images of a portion of a cardiovascular system, the method comprising: 

receiving from an image scanner a plurality of images recorded over a period of time, 
the images representing one or more locations along the extent of the cardiovascular system; 
and 

selecting at least a subset of the images based on common criteria determined from the 
plurality of images and without reference to an external signal. 

29. The computer-readable medium of claim 28, wherein the portion of the cardiovascular 
system is the heart. 

30. The computer-readable medium of claim 28, wherein the image scanner is a CT 
(Computer Tomography) scanner. 
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3 1 . The computer-readable medium of claim 28, wherein the image scanner is a MRI 
(Magnetic Resonance Image) scanner. 



32. The computer-readable medium of claim 28, wherein the image scanner is an 
Ultrasound scanner. 

33. The computer-readable medium of claim 28, wherein selecting a subset of the images 
results from a determination of the blurriness of each image. 

34. The computer-readable medium of claim 33, wherein the blurriness of the image is 
determined by a Fourier transform applied to the image. 

35. The computer-readable medium of claim 33, wherein the blurriness of the image is 
determined by the mean pixel difference between the image and an adjacent image. 

36. The computer-readable medium of claim 28, wherein selecting a subset of the images 
results from a determination of a change of a relative position of at least one vessel edge in 
each image. 

37. A computer-readable medium having computer executable instructions for performing 
a method for ordering a plurality of images of a portion of a cardiovascular system, the 
method comprising: 
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receiving from an image scanner a plurality of images recorded over a period of time, 
the images representing one or more locations along the extent of the cardiovascular system; 
deriving a cardiac cycle signal from the plurality of scanned images; and 
assigning a phase in the cardiac cycle to each scanned image. 

38. The computer-readable medium of claim 37, wherein the portion of the cardiovascular 
system is the heart. 

39. The computer-readable medium of claim 37, wherein the image scanner is a CT 
(Computer Tomography) scanner. 

40. The computer-readable medium of claim 37, wherein the image scanner is a MRI 
(Magnetic Resonance Image) scanner. 

41 . The computer-readable medium of claim 37, wherein the image scanner is an 
Ultrasound scanner. 

42. The computer-readable medium of claim 37, wherein deriving the cardiac signal 
comprises: 

segmenting a set of data representing a blood vessel in each image; 
computing a change value for the blood vessel; and 

determining the cardiac cycle signal based on a sequence of the change values for each 
image. 
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43. The computer-readable medium of claim 42, wherein the change value is a change in 
the area of a cross section of the blood vessel. 

44. The computer-readable medium of claim 42, wherein the change value is a change in 
the position of a wall of the blood vessel. 

45. The computer-readable medium of claim 37, wherein deriving the cardiac cycle signal 
comprises: 

segmenting a set of data representing a cross-section of the aorta in each image; 
computing an area value representing an area of the cross-section; and 
determining the cardiac cycle signal based on a sequence of the area values for each 
image. 

46. The computer-readable medium of claim 45, further comprising estimating the 
position of the aorta within the image data prior to segmenting the set of data. 

47. The computer-readable medium of claim 46, wherein estimating the position of the 
aorta utilizes a Hough transform. 



48. The computer-readable medium of claim 45, wherein segmenting of the aorta cross- 
section utilizes Dynamic Programming. 

49. The computer-readable medium of claim 45, further comprising filtering the cardiac 
cycle signal to produce a smoothed cardiac cycle signal. 

50. The computer-readable medium of claim 37, wherein deriving the cardiac cycle signal 
comprises: 

for each image in the plurality of images performing the tasks of: 
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selecting an adjacent subsequent image; 

calculating a mean pixel difference between the image and the subsequent 
image; and 

determining the cardiac cycle signal based on the mean pixel differences of the 

images. 

5 1 . The computer-readable medium of claim 37, wherein deriving the cardiac cycle signal 
comprises: 

segmenting a set of data representing a cross-section of a heart in each image; 
computing an area value representing an area of the cross-section; and 
determining the cardiac cycle signal based on a sequence of the area values for each 
image. 

52. The computer-readable medium of claim 37, wherein deriving the cardiac cycle signal 
comprises: 

for each image in the plurality of images performing the tasks of: 
determining a first border of a heart in the image; 
determining a second border of the heart in a subsequent adjacent image; 
determining the difference between the first border and the second border; and 
determining the cardiac cycle based on a sequence of the differences. 

53. The computer-readable medium of claim 37, wherein the ordered set of images is 
further 

filtered to produce a subset of images, said subset of images comprising images acquired at a 
desired point in the cardiac cycle signal 
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54. The computer-readable medium of claim 37, wherein the derived cardiac cycle signal 
is used to interpolate or reconstruct new images at specific phases in the cardiac cycle from 
the original scanned images or other related data. 
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ABSTRACT 

Systems and methods for deriving a cardiac cycle signal by selecting images of a 
portion of a cardiovascular system include receiving a plurality of images from a scanner that 
have been recorded over a period of time. The images represent one or more locations along 
5 the extent of the cardiovascular system. The images are then selected based on common 
criteria determined from the plurality of images and without reference to an external signal 
The common criteria comprises changes in the size of a cross section of the aorta, changes in 
the volume of the heart, changes in the area of a cross section of the heart. In addition, the 
criteria can include the mean pixel difference between adjacent images. 
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Schumm, Sherry W. Reg. No. 39,422 

Schwegman, Micheal L. Reg. No. 25,816 

Scott, John C. Reg. No. 38,613 

Smith, Michael G. Reg. No. 45,368 

Speier, Gary J. Reg. No. 45,458 

Steffey, Charles E. Reg. No. 25,179 

Terry, Kathleen R. Reg. No. 31,884 

Tong, Viet V. Reg. No. 45,416 

Viksnins, Ann S. Reg. No. 37,748 

Woessner, Warren D. Reg. No. 30,440 



in I hereby authorize them to act and rely on instructions from and communicate directly with the 
per^ii/assignee/attorney/firm/organization/who/which first sends/sent this case to them and by whom/which I hereby declare that I 
havelonsented after full disclosure to be represented unless/until I instruct Schwegman, Lundberg, Woessner & Kluth, P.A. to the 
confcllry. 

Please direct all correspondence in this case to Schwegman, Lundberg, Woessner & Kluth, P.A. at the address indicated below: 
1 P.O. Box 2938, Minneapolis, MN 55402 

C Telephone No. (612)373-6900 

i"y I hereby declare that all statements made herein of my own knowledge are true and that all statements made on information and 
belilPare believed to be true; and further that these statements were made with the knowledge that willful false statements and the like so 
maclOare punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that such willful false 
statements may jeopardize the validity of the application or any patent issued thereon. 



Full Name of joint inventor number 1 : Vincent J. Argiro 
Citizenship: United States of America 

Post Office Address: 1617 West Franklin Avenue 

Minneapolis, MN 55405 



Signature: 



Residence: Minneapolis, MN 




Date: 



Full Name of joint inventor number 2 : Marek Breil 
Citizenship: Czech Republic 

Post Office Address: 1 5720 Rockford Road, Apt. 2 1 1 

Plymouth, MN 55446 



Signature: 




Residence: Plymouth, MN 



Date: 



X Additional inventors are being named on separately numbered sheets, attached hereto. 
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Serial No. not assigned 

Filing Date; not assigned „ 

I hereby declare that all statements made herein of my own knowledge are true and that all statements made on information and 
belief are believed to be true; and further that these statements were made with the knowledge that willful false statements and the like so 
made are punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that such willful false 
statements may jeopardize the validity of the application or any patent issued thereon. 

Full Name of joint inventor number 3 : Renee Maheshwari Rashid 

Citizenship: United States of America Residence; Santa Clara, CA 

Post Office Address: 1821 Andrea Place 

Santa Clara, CA 95051 

Signature: Date: 

Renee Maheshwari Rashid 



Full Name of joint inventor number 4 : Todd Johnson 
Citizenship: United States of America 

Posuatffice Address: 2871 South Humboldt Ave, #21 

;5 Minneapolis, MN 55408 



Signffiire: 



/ Todd Johnson 



Residence: Minneapolis, MN 



Date: SepW^W^ *ot. jdtofeO 



FuUjfame of joint inventor number 5 : Milan Breil 
Citizenship: Czech Republic 

Post-Office Address: Kulturni 1766 

£3 Rozmov pod Radhostem 

5 756 61 

Czech Republic 



Signature: 



Milan Brejl 



Residence: Czech Republic 



Date: 



Full Name of inventor: 

Citizenship: Residence: 
Post Office Address: 



Signature: 



Date: 



Sent by: -RENDITION 
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Serial No. not assigned 
Filing Date; not assigned 



I hereby declare that all statements made herein of my own knowledge are true and that all statements made on information and 
belief are believed to be true; and further that these statements were made with the knowledge that willful false statements and the like so 
made are punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that such willful false 
statements may jeopardize the validity of the application or any patent issued thereon. 

Full Name of joint inventor number 3 : Renee Maheshwari Raahid 

Citizenship; United States of America Residence: Santa Clara, CA 

Post Office Address: 1821 Andrea Place 

Santa Clara, CA 95051 



eipe-Maheshwari Rash id 1 i -r~ > Tfj 7 



Signature: Dat e: ^f/ J2-//7Q 



Full Name of joint inventor number 4 : Todd Johnson 

Citizenship: United States of America Residence: Minneapolis, MN 

Post Office Address : 287 1 South Humboldt Ave, #2 1 

Minneapolis, MN 55408 

Signature: Date: 

Todd Johnson 



Full NaSe of joint inventor number 5 : Milan Brejt 

Citizensfiip: Czech Republic Residence: Czech Republic 

Post Office Address: Kulturni 1766 

Rozmov pod Radhostem 

J. 756 61 

Czech Republic 

Signature: Date: 
" ; ; Milan Brejl 



Full Name of inventor 

Citizenship: Residence: 
Post Office Address: 



Signature: 



Date: 
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Senal No. not assigned 

Filing Date: not assigned 

I hereby declare that all statements made herein of my own knowledge are true and that all statements made on information and 
belief are believed to be true; and further that these statements were made with the knowledge that willful false statements and the like so 
made are punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that such willful false 
statements may jeopardize the validity of the application or any patent issued thereon. 

Full Name of joint inventor number 3 : Renee Maheshwari R flrhil 1 

Citizenship: United States of America Residence: Santa Clara, CA 

Post Office Address: 1821 Andrea Place 

Santa Clara, CA 95051 

Signature: Date: 

Renee Maheshwari Rashid 



Full Name of joint inventor number 4 : PttW Jfrfrp^n 
Citizenship: United States of America 

Post Office Address: 2871 South Humboldt Ave, #21 

Minneapolis, MN 55408 



Residence: Minneapolis, MN 



Signature:- 



Todd Johnson 



Date: 



Full Name^f joint inventor number 5 : Milan fl reil 



Citizenship: 

Post Office Address: 



Signature:^ 



Czech Republic 
Kulturni 1766 
Rozmov pod Radhostem 
756 61 

Czech Republic 



Milan Brejl 



Residence: Czech Republic 



Date: 



Full Nanie of inventor: 

Citizenship: 

Post Offifce Address: 



Residence: 



Signature: 



Date: 
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§ 1.56 Duty to disclose information material to patentability. 

(a) A patent by its very nature is affected with a public interest. The public interest is best served, and the most effective patent 
examination occurs when, at the time an application is being examined, the Office is aware of and evaluates the teachings of all information 
material to patentability. Each individual associated with the filing and prosecution of a patent application has a duty of candor and good 
faith in dealing with the Office, which includes a duty to disclose to the Office all information known to that individual to be material to 
patentability as defined in this section. The duty to disclose information exists with respect to each pending claim until the claim is canceled 
or withdrawn from consideration, or the application becomes abandoned. Information material to the patentability of a claim that is canceled 
or withdrawn from consideration need not be submitted if the information is not material to the patentability of any claim remaining under 
consideration in the application. There is no duty to submit information which is not material to the patentability of any existing claim. The 
duty to disclose all information known to be material to patentability is deemed to be satisfied if all information known to be material to 
patentability of any claim issued in a patent was cited by the Office or submitted to the Office in the manner prescribed by §§ L97(b)-(d) and 
1.98. However, no patent will be granted on an application in connection with which fraud on the Office was practiced or attempted or the 
duty of disclosure was violated through bad faith or intentional misconduct. The Office encourages applicants to carefully examine: 

(1) prior art cited in search reports of a foreign patent office in a counterpart application, and 

(2) the closest information over which individuals associated with the filing or prosecution of a patent application believe any 
pending claim patentably defines, to make sure that any material information contained therein is disclosed to the Office. 

(Bp Under this section, information is material to patentability when it is not cumulative to information already of record or being 
macMJof record in the application, and 

dj (1) It establishes, by itself or in combination with other information, a prima facie case of unpatentability of a claim; or 

T (2) It refutes, or is inconsistent with, a position the applicant takes in: 

T S (i) Opposing an argument of unpatentability relied on by the Office, or 

IS (ii) Asserting an argument of patentability. 

A gMma facie case of unpatentability is established when the information compels a conclusion that a claim is unpatentable under the 
prejlehderance of evidence, burden-of-proof standard, giving each term in the claim its broadest reasonable construction consistent with the 
specification, and before any consideration is given to evidence which may be submitted in an attempt to establish a contrary conclusion of 
patentability. 

(c) Individuals associated with the filing or prosecution of a patent application within the meaning of this section are: 

(1) Each inventor named in the application: 

(2) Each attorney or agent who prepares or prosecutes the application; and 

(3) Every other person who is substantively involved in the preparation or prosecution of the application and who is associated 
with the inventor, with the assignee or with anyone to whom there is an obligation to assign the application. 

(d) Individuals other than the attorney, agent or inventor may comply with this section by disclosing information to the attorney, 
agent, or inventor. 



